Skip to content

fix(plugin): filter non-object JSONL lines in Store.read_lines and accounting.summarize - #200

Merged
Aaron Li (aaron-jx-li) merged 2 commits into
microsoft:mainfrom
TejasLamba2006:fix/isinstance-ledger-records
Nov 16, 2026
Merged

Aaron Li (aaron-jx-li) merged 2 commits into
microsoft:mainfrom
TejasLamba2006:fix/isinstance-ledger-records

Conversation

@TejasLamba2006

@TejasLamba2006 Tejas Lamba (TejasLamba2006) commented Nov 10, 2026 •

Copy link
Copy Markdown
Contributor

Summary

Store.read_lines could return non-dict JSON values (null, arrays, strings) if a JSONL log had a malformed line, which crashed callers calling .get() and took slots from limit.

  • Filter decoded lines with isinstance(rec, dict) inside Store.read_lines so all callers receive object records and limit counts valid records only.
  • Keep isinstance(r, dict) at entry of accounting.summarize for direct list callers.
  • Add unit tests in tests/test_store_read_lines.py.

Test plan

  • pytest tests/test_store_read_lines.py
  • Full tests/ suite passes

…repos

store.read_lines can return non-dict JSON values (null, arrays, strings)
if a ledger file was corrupted or written incorrectly. Code calling .get()
on these records crashes with AttributeError.

Added isinstance(r, dict) checks before accessing record fields in:
- accounting.summarize (filters at entry)
- repos._infer_repo_from_state (inline check)
- repos._merge_state (filters ledger lists, inline check for events)

Same pattern as the distiller fix in microsoft#199.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I checked every caller of Store.read_lines at the PR head (git grep "read_lines("). There are six call sites. cli.py:215 passes its records to accounting.summarize, which this PR now filters. The PR also guards repos.dir_label and the three reads in repos._merge_host. history._session_records already had the check. So no caller is left unguarded. In _merge_host, the filtered d_ledger is what _write_lines writes back, so a merge also drops the corrupt lines from the destination ledger. That seems like the right outcome.

Two small things:

  1. One guard in Store.read_lines would replace all four. If line 112 of store.py only appends a decoded line when isinstance(rec, dict), every current and future caller is covered. The limit slice would then count only real records; today a corrupt line near the end takes one of the limit slots. The per-caller checks could then go.
  2. The description names _infer_repo_from_state and _merge_state, but the changed functions are dir_label and _merge_host. Also, the PR adds no test, and the checked boxes don't say what was run. A test that writes a ledger with a null line and calls summarize would pin it.

Reviewed with help from Claude Code.

A non-object JSONL line (null, a list, a string) came back as a
record, so a trailing null with limit=1 hid the last valid record and
callers crashed on it. Filter to dict records before the limit slice,
so every reader gets dicts and the inline checks in repos.py are not
needed. accounting.summarize keeps its own filter for direct lists.
@TejasLamba2006 Tejas Lamba (TejasLamba2006) changed the title fix(plugin): guard against non-dict ledger records in accounting and repos fix(plugin): filter non-object JSONL lines in Store.read_lines and accounting.summarize Nov 14, 2026
@TejasLamba2006

Copy link
Copy Markdown
Contributor Author

Thanks Muhammad Abdullah Butt (@1AbdullahButt), good call moving the filter into Store.read_lines. Pushed 25ae385: Store.read_lines now filters to dict records before slicing limit, repos.py is back to upstream, accounting.summarize keeps its entry guard, and tests/test_store_read_lines.py covers both. Updated the PR description as well.

@aaron-jx-li
Aaron Li (aaron-jx-li) merged commit 8f6cd51 into microsoft:main Nov 16, 2026
9 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants